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ABSTRACT : 

Conditions  are  given  which  ensure  that  a  recently  developed 
exact  penalty  function  algorithm  for  nonlinear  programming  problems 
is  well  defined. 

Prepared  by: 


I.   THE  PROBLEM 

We  consider  the  nonlinear  programming  problem  with  inequality 
constraints 

NLP    maximize    f(x) 

subject  to   g.(x)  ^  0    i  =  l,...,m  (1) 

x  e  En 

where   f  and   g.   are  continuously  dif ferentiable  and  for  simplicity 
we  assume  that  a  constraint  qualification  holds.   In  reference  [1]  we 
developed  a  penalty  function  algorithm  which  involves  alternate  cycles 
of  1)  maximizing  a  penalty  function  and  2)  approximating  optimal  Lag- 
range multipliers.   In  the  interests  of  brevity  the  reader  is  referred 
to  [lj  for  a  complete  description  of  the  procedure  and  its  convergence 
properties.   For  the  purposes  of  this  report  it  is  enough  to  recall  that 
at  each  maximization  cycle  we  have  fixed  k  >  0  e  E*  ,   A  >  0  e  E 
and  we  want  to  find  x  e  E   to  maximize  the  penalty  function 

m 
P(x,k,A)  =  f(x)  -  (1/k)  I   A.{exp(kg  (x))-l]  (2) 

i=l 

In  [2]  Gould  and  Howe  show  that  under  certain  weak  conditions  if  x* 
is  a  local  maximum  for  NLP(l)  with  optimal  Lagrange  multipliers   A*  , 
then  there  exists  a  k  sufficiently  large  that  P(x,k,A*)   has  a  local 
maximum  at  x*  .   Thus,  if   A*   is  known  and  k  is  chosen  large  enough, 
the  function  P(x,k,A*)   is  an  exact  penalty  function.   The  algorithm 
in  [1]  attempts  to  determine  the  optimal  Lagrange  multipliers  iteratively 
while  simultaneously  searching  for  x*  .   Thus,  it  involves  maximizing 


P(x,k,A)   for  fixed   A  ^  A*  ,  and  it  is  important  to  determine  conditions 
under  which  this  maximization  can  be  accomplished.   If,  for  the  particu- 
lar choice  of   A  being  used  at  some  cycle,  the  P(x,k,A)   function  is 
unbounded  (as  a  function  of  x  )   then  the  algorithm  is  not  well  defined. 


II.   CONDITIONS  FOR  A  WELL  DEFINED  ALGORITHM 

In  this  section  we  give  two  sets  of  conditions  and  prove  that 
under  either  set  of  conditions  the  penalty  function  has  a  finite  global 
maximum  for  any  k  >  0  e  E1   and  any  X  >  0  e  E 

Theorem  1:   If   f  (x)  •*»  -  «  as    |x|  |  ■+   °°  then  the  algorithm  is  well 
defined. 

Proof:   The  proof  is  simple  and  requires  only  that  we  examine  the  penalty 
term  for  the   ith   constraint 


-(1/k)   A.texpCkg.Cx))-!]  (3) 


with     k  >   0    ,    X.    >   0    .      As      g    (x)   ->  «      (infeasible)  ,      (3)   -»-   -  °°      ;   but 
as      g    (x)  ->  -  »      (feasible)      (3)  ->  X./k   . 

Thus, 

m  m 

-    (1/k)      I  X.    exp[(kg.(x))-l]   <:     I   X    /k  (4) 

i=l   X  X  i=l 

So  if   f  (x)  ■>  -  oo   for  all  x  with    |x|  |   •*  °°  ,  then  also 
P(x,k,X)  ->  -  •  as    |x||  ->■  °°  .   Hence,  P   is  not  unbounded  for  maximi- 
zation, and  this  is  true  for  any  k  >  0  e  E1   and  any  X  £  0  e  E   .   I 1 

Theorem  2:   If  a)  f(x)   is  a  strictly  concave  function 

b)  g.(x)   are  convex  functions  for  all   i  =  l,...,m,  and 


c)  NLP(l)   has  finite  optimal  solution  x*  with  optimal 
Lagrange  multipliers   X* 

then  for  any  fixed  k>0eE1,X>0eE   ,3  a  finite  x  which 
maximizes  P(x,k,A). 

Proof;   For  fixed  k  >  0  ,  X  >  0  we  know  that  P(x,k,X)   is  a  strictly 
concave  function  of  x  because  of  assumptions  a)  and  b) .   To  show  that 
a  finite  maximum  exists  it  suffices  to  show  that  a  finite  maximum  along 
any  half  line  exists,  that  is,   V  x  ,  d  e  E   ,   as   9  e  E1  -►  °°   the 
derivative 

P'O)  =  -~-  P(x+6d,k,X)  (5) 

must  eventually  become  negative  (or  equal  zero). 

Since  x*  maximizes  P(x,k,X*)  (see  {2J)  we  know  this  is  true 
for  A  =  X*  . 

Thus,    consider   any     x,deEn,k>0eE1,X>0eE      . 

P'O)    =  ~Jq-     P(x+6d,k,A) 

|f(x+6d)    -    (1/k)      I   Xi[exp(kgi(x+ed))-lj|  (6) 


de 


Xi 


=  f(e)  -    I  (4M  exp[kg.(e)]  g.'(e) 
i=i     K 

where  we  have  introduced  the  abbreviations   f'(8)  =   ,    f(x+6d)  , 

g±(e)  =  g1(x+ed)  ,  g.'(e)  e  -A-  gi(x+ed)  . 


There  are  several  cases  for  each  of   f  and  g   : 


Cases  for  f   (concave) 

f  1    f  (6 )  ->  -  oo  as   e+°° 

f  2    f(9)   is  non  decreasing  as   9  ->■  » 

then  f'(9)  >  0  and  decreasing  as  9  ->  °°  (7) 

Cases  for  each  g.  (convex) 

gl    g  (9)  ■+  °°  as   9  -+  °° 

then  'Jr  >  0  such  that  eventually  g.'(9)  ^  r   so  that 

-  -~~  [exp  kg. (9)]  g.'(9)  -  -  oo   as   9  +   »  (8) 


g_2    g,(9)   is  non  increasing  as   9  ->•  °°  then  exp  kg.  (9)  >  0 

and  non  increasing  as   9  ->  °°    ,  g . '  (9)  £  0  and  non  decrea- 
sing as   9  ■*■  °°  ,   so — [exp  kg.  (9)]  g  '  (9)  ^  0  and 

non  increasing  as   0  ->  °°  .  (9) 


Analysis  of  Cases 

1.  If   f^l  holds,  then  P(0)  -*  -  °°  as   9  +  °°  for  any  combina- 
tion of  cases  on  the   g.  .   (see  Theorem  1)   Thus,   P  has 

a  finite  global  maximum  for  any  X   >   0  . 

2.  If   f2  holds, 

2a)  If  also  3i  with  g   in  case  _gl_  then  (7)  ,  (8)  ,  (9) 


show  that  P'(9)   in  (6)  approaches  -  °°  as   9  ->  °°  . 
Thus  P'(9)   must  eventually  become  negative  so  that 
P  has  a  finite  max  along  the  half  line. 
2b)  Otherwise  for  all   i  =  l,...,m  constraint   g.   is  in 
case  g2. 

then  (6),  (7)  and  (9)  show  that  P'(9)  >  0  as  9  ■>  - 
so  that  no  maximum  is  attained.  But  this  is  true  for 
any   X   including  X   =  A*  . 

Thus,   P(x,k,X*)   does  not  have  a  finite  maximum  contrary 
to  assumption.   Hence  case   2b   cannot  occur  if  NLP 

n 

has  a  finite  optimal  solution.  I 1. 


III.   SUMMARY 

Two  sets  of  conditions  which  guarantee  that  the  exact  penalty 
function  algorithm  in  [lj  is  well  defined  have  been  presented  and  proved. 
It  should  perhaps  also  be  emphasized  that  the  algorithm  in  [1]  is  pri- 
marily a  method  for  locating  local  solutions  to  NLP  (1) ,  and  that  it  can 
frequently  do  this  even  if  P(x,k,X)   is  globally  unbounded,  (for  non 
concave  problems)  since  in  the  region  of  search  P(x,k,A)   will  still 
have  a  local  maximum.   In  such  a  case,  however,  the  possibility  of 
straying  into  the  global  unbounded  region  cannot  be  overlooked. 
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